Add hid_set_num_input_buffers() API by auxcorelabs · Pull Request #787 · libusb/hidapi

auxcorelabs · 2026-04-17T08:55:21Z

Exposes a new public function hid_set_num_input_buffers(dev, num_buffers) to resize the per-device input report queue.

High-throughput HID devices (medical telemetry, high-poll-rate gaming peripherals, data acquisition hardware) can emit bursts of input reports that exceed the current hardcoded queue sizes (30 on macOS and Linux/libusb, 64 on Windows). When a burst exceeds the queue, reports are silently dropped with no error indication to the caller. Observed on hardware emitting ~90 reports in 200 ms bursts: both the macOS and Linux/libusb backends silently drop ~20% of frames.

Per-backend behavior

macOS: resizes the userspace input-report queue
Linux libusb: resizes the userspace report queue
Windows: wraps HidD_SetNumInputBuffers (parity with existing behavior)
Linux hidraw / NetBSD: no-op (kernel manages buffering — documented as a @note on the API declaration in hidapi.h)

Safety

Values outside [1, HID_API_MAX_NUM_INPUT_BUFFERS] return -1 and register a descriptive error on the device (via the backend's register_device_error helper where available)
Thread-safe: setter uses the same lock (dev->mutex / dev->thread_state) as the respective report callback

Design notes

Backwards compatibility: Purely additive public API. Defaults per backend are unchanged, so existing callers see no behavioral change. hid_device is opaque in hidapi.h, so struct field additions in backend-private files are not an ABI break.
Why 1024 cap: Bounds per-device memory to roughly 1 MB on USB high-speed links, preventing memory exhaustion from a caller that passes INT_MAX (malicious or buggy). We've measured bursts up to ~90 reports deep in practice; 1024 leaves room for devices ~10× more demanding. Overridable at build time via -DHID_API_MAX_NUM_INPUT_BUFFERS=N for memory-constrained targets.
No getter: hidraw and NetBSD back the queue in the kernel with sizes not portably exposed to userspace, so a cross-platform getter couldn't return a meaningful value there.
Why not hid_get_input_report() instead? That API issues a host-initiated GET_REPORT request (via the control endpoint or OS equivalent). It does not drain the interrupt-endpoint queue that streaming devices fill, and in practice many devices either don't implement GET_REPORT for input reports or respond incorrectly. The two APIs use different USB transfer mechanisms (interrupt endpoint vs. control endpoint); this PR fixes the interrupt-streaming path.

Userspace queue: flat pre-allocated ring buffer

The userspace queue is a fixed-size ring with inline-slot storage:

Pre-allocated, not MAX-sized. Backing store is two exact-size allocations — a slot data buffer of (dev->num_input_buffers × slot_size) bytes plus a parallel lengths[] array — both allocated once at device open. No MAX_NUM_INPUT_BUFFERS over-allocation.
slot_size determined at runtime per backend:
- macOS — kIOHIDMaxInputReportSizeKey (IOKit-parsed HID report descriptor; authoritative)
- libusb — interrupt-IN wMaxPacketSize (matches the existing single-packet transfer sizing)
- 64-byte clamp on both backends as a defensive fallback when the OS reports no value
O(1) push/pop. Push is memcpy-only (no per-report malloc); pop is copy-out (pop_into(r, dst, dst_len)) — the ring owns its storage end-to-end, no borrowed-pointer contract.
Resize. hidapi_input_ring_resize() allocates new storage, memcpys survivors in FIFO order, frees old, swaps under the caller-held mutex. Shrink-below-count drops oldest, matching push-time policy.
Implementation. Single shared header core/hidapi_input_ring.h; backend .c files include it as a static-in-header helper.
Tests. tests/test_hidapi_input_ring.c — test cases covering init/destroy idempotency, push/pop FIFO, drop-oldest, varying lengths, resize grow/shrink/wrap/empty/post-wrap/invalid-args, plus a 10k-iteration stress run.

References

when I use hidapi, how to use 'HidD_SetNumInputBuffers' to set Buffer size ? #154 — this PR also addresses the prior request to expose HidD_SetNumInputBuffers functionality to callers

Exposes a new public function to resize the per-device input report queue. High-throughput HID devices (medical telemetry, high-poll-rate gaming peripherals, data acquisition hardware) emit bursts of input reports that exceed the current hardcoded queue sizes (30 on macOS and the libusb backend, 64 on Windows). When a burst exceeds the queue, reports are silently dropped with no error indication to the caller. This adds: - hid_set_input_report_buffer_size(dev, size) in hidapi.h - HID_API_MAX_INPUT_REPORT_BUFFER_SIZE (1024) cap to prevent unbounded memory growth Per-backend behavior: - macOS: resizes the userspace IOHIDQueue-fed report queue - Linux libusb: resizes the userspace report queue - Windows: wraps HidD_SetNumInputBuffers (parity with existing behavior) - Linux hidraw: no-op (kernel manages buffering) - NetBSD: no-op (kernel manages buffering) Defaults are unchanged, so existing callers are unaffected. Values outside [1, HID_API_MAX_INPUT_REPORT_BUFFER_SIZE] are rejected with -1. Thread-safe on macOS (dev->mutex) and libusb (dev->thread_state), matching the locks used by the respective report callbacks. Addresses the same need as closed issue libusb#154 (HidD_SetNumInputBuffers exposure) and complements libusb#725 (callback-based input API).

Youw

refer to comments, otherwise looks good

Per maintainer feedback on PR libusb#787: - Remove if (!dev) validation from all 5 backends. hidapi convention is that device functions trust the caller to pass a valid handle; only hid_close is permitted to accept NULL. - Reword the inline comment in linux/hid.c and netbsd/hid.c to lead with "No-op" so the caller-visible behavior is explicit at the implementation site.

auxcorelabs · 2026-04-17T16:21:36Z

Thanks for reviewing. I have made the changes.

JoergAtGithub · 2026-04-18T11:41:56Z

I think the name of the Win32-API HidD_SetNumInputBuffers describes better what it does than hid_set_input_report_buffer_size, as it does change the number of buffers and not the size of the bufferin bytes.

… controls the number of input report buffers, not their byte size. - Function: hid_set_input_report_buffer_size -> hid_set_num_input_buffers - Macro: HID_API_MAX_INPUT_REPORT_BUFFER_SIZE -> HID_API_MAX_NUM_INPUT_BUFFERS - Parameter: buffer_size -> num_buffers - Error string: "buffer_size out of range" -> "num_buffers out of range"

auxcorelabs · 2026-04-19T05:59:20Z

Good callout @JoergAtGithub. Have renamed the function to hid_set_num_input_buffers() and made the associated changes.

@Youw - Please let me know if any further changes required.

Thanks both.

mcuee · 2026-04-19T08:01:47Z

@auxcorelabs
Just wondering if you can enhance the hidtest application to include this new API.

If not, do you have a simple test to share? Thanks.

@note

…ile commentary, add hidtest coverage - hidapi/hidapi.h: replace the Defaults per backend list with an @note Per-backend behavior block covering macOS / Windows / libusb / hidraw / uhid semantics, ranges, and defaults. Per @Youw, the public header is the canonical place for the cross-backend contract. - linux/hid.c, netbsd/hid.c: drop the comment that cross-referenced other backends. The (void)num_buffers; idiom and the header contract speak for themselves. - libusb/hid.c: drop the self-scoped no-error-registration note for the same reason. - hidtest/test.c: add a compile-time symbol reference and a runtime call hid_set_num_input_buffers(handle, 500) right after hid_open() succeeds, per @mcuee. Both guarded on HID_API_VERSION >= 0.16.0 so they activate in the 0.16 release cycle, matching the precedent of hid_send_output_report at 0.15.0.

Youw

So far so good.

Copilot

Pull request overview

Adds a new public HIDAPI entry point to let callers increase the per-device input report queue depth, addressing silent report drops during bursty/high-throughput input on backends that queue reports in userspace (macOS + Linux/libusb) and exposing the existing Windows kernel buffering control.

Changes:

Introduces hid_set_num_input_buffers(dev, num_buffers) and documents it in hidapi.h (with a max-cap macro).
Updates macOS and Linux/libusb backends to enforce queue limits via a per-device num_input_buffers value (default 30).
Adds no-op stub implementations for Linux hidraw and NetBSD uhid; updates hidtest to exercise the new API when available.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`hidapi/hidapi.h`	Declares and documents the new public API and its max-cap macro.
`mac/hid.c`	Adds per-device buffer cap field, uses it in the report callback, and provides a setter guarded by the device mutex.
`libusb/hid.c`	Adds per-device buffer cap field, uses it in the read callback, and provides a setter guarded by the thread-state mutex.
`windows/hid.c`	Implements the API by forwarding to `HidD_SetNumInputBuffers`.
`linux/hid.c`	Adds a validated no-op stub for the hidraw backend.
`netbsd/hid.c`	Adds a validated no-op stub for the uhid backend.
`hidtest/test.c`	Calls the new API under a version guard to ensure all backends export it.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Per-backend helper consistency: * linux/hid.c, netbsd/hid.c, mac/hid.c: setter uses register_device_error() instead of register_error_str() directly. Build-time override: * hidapi/hidapi.h: wrap HID_API_MAX_NUM_INPUT_BUFFERS in #ifndef so downstreams can set the cap via -DHID_API_MAX_NUM_INPUT_BUFFERS=<value>. Ring buffer input queue: * New static-in-header helper hidapi_input_ring_*, present in libusb/ and mac/ as byte-identical copies. * libusb/hid.c, mac/hid.c: replace struct input_report * linked list with fixed-size ring. Enqueue is O(1); eviction is inline in push; the setter shrinks via drop_oldest so dev->num_input_buffers is the exact steady-state cap. * ABI unchanged (hid_device is opaque in hidapi.h). * Allocation failure in the read callback is now handled — the previous code had an unchecked malloc() that would segfault. libusb has no active error channel so the drop is silent there; mac calls register_device_error.

Youw

This is an impressive amount of new code contributed in a short amount of time.
What AI tool has been used to generate it? I will not believe it was hand-written.

auxcorelabs · 2026-04-23T15:08:37Z

This is an impressive amount of new code contributed in a short amount of time. What AI tool has been used to generate it? I will not believe it was hand-written.

This was a relatively large change, developed using a structured workflow with human oversight & guidance. The design and approach/choices were planned and reviewed (via AI-assisted passes and manually) before any code change. Two AI models were then used in complementary roles—one for code drafting and the other for iterative review to refine the result. The output went through human review and feedback, and the final changes were manually validated, including building across available platforms to verify there were no errors before committing.

From my experience, the effectiveness comes primarily from the structured workflow and repeated AI review iterations, followed by human review & guidance.

Youw · 2026-04-23T17:03:07Z

the effectiveness comes primarily from the structured workflow and repeated AI review iterations, followed by human review & guidance

I generally don't mind as I'm using similar tools and aproaches myself in various other projects.

But since HIDAPI is open-source well-know public repo with many users and this is a first huge contribution to HIDAPI using AI-tools, I'd like to mention this explicitly in a form suggested by https://docs.kernel.org/process/coding-assistants.html, specifically the commit message/PR description attribution (at least I'm planning to include it in the final squash-merge commit into master), e.g.:
Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

that's why I asked what AI tool/model was used to produce this.

I'll make a generic NOTE about this somewhere in the README a bit later.

Switch ring storage from pointer-slot (per-push malloc) to a single flat buffer of (dev->num_input_buffers × slot_size), allocated at device open. slot_size: kIOHIDMaxInputReportSizeKey on macOS, interrupt-IN wMaxPacketSize on libusb, 64 B fallback. Resize re-allocs and memcpys survivors FIFO-order. Pop becomes copy-out (pop_into). Ring header consolidated into core/. Adds tests/test_hidapi_input_ring.c — 21 ASAN tests.

auxcorelabs · 2026-04-25T13:20:45Z

Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

Thanks for the information. Please find details below:
Assisted-by: Claude Code CLI:claude-opus-4.6
Assisted-by: Claude Code CLI:claude-opus-4.7
Assisted-by: Codex CLI:gpt-5.4

Youw · 2026-04-25T13:35:42Z

+	if (push_rc == 0) {
+		pthread_cond_signal(&dev->condition);
+	} else {
+		register_device_error(dev, "input queue allocation failed");


as per thread-safety notes posted on Wiki - register_device_error itself is not thread-safe (and not intended to be) and accessing error string from different threads (hid_report_callback thread in this case) can cause a data race

Youw · 2026-04-25T13:40:37Z

+        return -1;
+
+    r->storage = (uint8_t *)malloc((size_t)capacity * slot_size);
+    r->lengths = (size_t *)calloc((size_t)capacity, sizeof(size_t));


I believe this doesn't have to be a separate allocation either. There could be a single storage and two non-owning poiinters to lengths and data

Youw

This looks great, thanks!

I did a quick review, but I want to go over it again what I have a bit more free time than now.

Thanks for the contribution! I'll have it merged some time soon.

mcuee added API API change, Version 1 stuff enhancement New feature or request labels Apr 17, 2026